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Abstract 

Existing literature on source coding with side information (SCSI) mostly uses the state-of- 
the-art channel codes namely LDPC codes, turbo codes, and their variants and assume classical 
unique decoding. In this paper, we present an algebraic approach to SCSI based on the list 
decoding of the underlying channel codes. We show that the theoretical limit of SCSI can 
be achieved in the proposed list decoding based framework when the correlation between the 
source and side information is g-ary symmetric. We argue that, as opposed to channel coding, 
the correct sequence from the list produced by the list decoder can effectively be recovered in case 
of SCSI with a few CRC symbols. The CRC symbols, which allow the decoder to identify the 
correct sequence, incur negligible overhead for large block lengths. More importantly, these CRC 
symbols are not subject to noise since we are dealing with a virtual noisy channel rather than 
a real noisy channel. Finally, we present a guideline for designing constructive SCSI schemes 
for non-binary and binary sources using Reed Solomon codes and BCH codes, respectively. 
This guideline allows us to design a SCSI scheme for any arbitrary g-ary symmetric correlation 
without resorting to simulation. 

1 Introduction 

Recently, in the context of sensor networks and mobile multimedia applications [551 [321 [13] , dis- 
tributed source coding has gained significant attention from the research community. Distributed 
source coding (DSC) (TU] refers to the compression of correlated sources that are not co-located. 
Thus in a DSC setting, the encoding of correlated sources is performed independently while decod- 
ing is done jointly. The information theoretic Umits for independent encoding of correlated sources 
have been established by Slepian and Wolf in f5B^. According to the Slepian-Wolf theorem, inde- 
pendent encoding of correlated sources with joint decoding can be as efficient as joint encoding and 
decoding. More specifically, in compressing two correlated sources X and Y, the rates achievable 
with independent encoding but joint decoding are bounded by Rx > H{X\Y), Ry > H{Y\X), and 
Rx + Ry > H{X,Y). In this paper, we focus on the asymmetric approach where Y is encoded 
at a rate H{Y) in the conventional way and X is encoded at a rate H{X\Y) assuming that Y is 
available at the decoder. This asymmetric approach is known as source coding with (decoder only) 
side information (SCSI) in the literature. 

The essential technique for source coding with decoder only side information is binning. Consider 
the encoding of a source X in the presence of the side information Y available only at the decoder. 

*M. All and M. Kuijpcr arc with the Department of Electrical and Electronic Engineering, University of Melbourne, 
VIC 3010, Australiamortuza94@gmail.com; mkuijper@unimelb.edu.au 

tThis work was presented in part in ISIT 2010. 
This work was supported by the Australian Research Council(ARC). 



1 



Let X and Y be the alphabets of X and Y respectively. For large enough n, with high probability, 
a source sequence x £ X" belongs to a set of approximately 2'^-^^-^^^'> sequences that are jointly 
typical with the side information sequence y G Y". Thus if y were available both at the encoder 
and decoder, the outcomes from the source X could be encoded using approximately H{X\Y) bits 
on average with a very small probability of error. In this case, both the encoder and decoder could 
construct the same set of jointly typical sequences and use the same indexing. Then encoding of a 
source sequence x with its index would lead to correct decoding. However, even if Y is not available 
at the encoder, it is possible to achieve the same rate of H{X\Y) using the technique of binning [9]. 
The idea is to randomly assign each of the source sequences in X" to one of the 2"^ bins, where 
R > H{X\Y). Given a source sequence x, the encoding operation is to transmit the index of the 
bin to which x belongs. The decoding operation is to choose the sequence x from the indexed bin 
which is jointly typical with the side information sequence y. Since, for large enough n, almost all 
the sequences that are jointly typical with a given y will belong to different bins, x will be equal to 
X with high probability. 

Although the Slepian-Wolf theorem states the fundamental limit on achievable compression, a prac- 
tical coding scheme to achieve this limit does not follow immediately. However, it follows from the 
above binning scheme that a practical binning algorithm needs to partition the source data space 
into a minimum number of bins while ensuing correct decoding. In other words, it should put as 
many source sequences as possible in a bin while ensuring that for any typical side information 
sequence each of the bins contains only one jointly typical source sequence. Thus each of the bins 
can play the role of a good channel code. This connection between binning and channel codes was 
first indicated in [32] while interpreting the Slepian-Wolf coding. 

Due to this close connection between binning and channel coding, most of the SCSI schemes proposed 
in the literature are based on the state-of-the-art channel codes namely LDPC codes, turbo codes, and 
their variants. Although their performances have been reported to be close to the theoretical limits, 
there are several obstacles in using them in practice. Firstly, the LDPC code based schemes [HJ [551 
[23] and turbo code based schemes [351 [TH [U [211 [T] niostly focus on binary sources. Although some 
compression schemes for non-binary sources have been proposed [35l [H [36] , they mainly map the 
non-binary symbols to fixed-length bit sequences and then encode the bit sequences using binary 
coding constructs. However, this requires decomposing the symbol level correlation into bit level 
correlation. 

Secondly, the near optimal performance of LDPC and turbo codes, as channel codes, can be at- 
tributed to their efficient soft-decision decoding algorithms. In soft-decision decoding, the soft out- 
put from a channel is directly fed into the decoder without demodulating to discrete values. Since 
demodulation of the received signal to hard values results in irretrievable loss of information, the 
soft-decision decoding results in approximately 2dB gain over its hard-decision decoding counter- 
parts. However, it follows from the binning framework that the virtual channel essentially models 
the correlation between the source and side-information and thus dictates the choice of the channel 
code that induces an efficient binning. Clearly, the processes of modulation and demodulation are 
not involved in the SCSI. Therefore, in LDPC and turbo code based SCSI schemes, the soft-decision 
coding gain is not readily available. 

Finally, there is always a gap between the compression rate that can be achieved with a specific 
channel code and the conditional entropy for which it can yield near lossless compression. For 
example, the turbo code based scheme in [4 has a compression rate of 0.67 but can achieve near 
lossless compression only when the true conditional entropy is H{X\Y) = 0.49. Similarly the LDPC 
code based scheme in [21], that achieves near lossless compression when H{X\Y) — 0.20, has an 
effective compression rate of 0.25. However, for these schemes there is no algebraic approach to 
choose a code of a particular rate, given a conditional entropy, that can ensure near lossless recovery. 
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Instead simulation is used to design a code, given the conditional entropy, due to the lack of algebraic 
construction of LDPC and turbo codes. 

In this paper, we present an algebraic approach to SCSI based on list decoding of the corresponding 
channel code that induces the partitioning of the source space. We demonstrate that for q-ary 
symmetric correlation between the source and side information, the list decoding approach can 
achieve optimal compression. Although list decoding yields a list of sequences, as opposed to the 
classical unique decoding, we show that in case of SCSI, the correct sequence can be effectively 
extracted from the list at a negligible increase in the optimal compression rate. One of the advantages 
of using list decoding is that it can reduce the compression rate significantly as compared to its 
classical unique decoding counterpart. Besides, this algebraic approach allows us to design a SCSI 
scheme for any arbitrary q-ary symmetric correlation without resorting to simulation. 

Although there are extensive research activities in the area of list decoding and the same holds for 
SCSI, only a couple of works [TT] can be found in the literature on the usage of list decoding for 
source coding problems. The use of list decoding has recently been proposed by Draper and Martinian 
in [TT for compound conditional source coding (CCSC). As a variant of Slepian-Wolf coding, CCSC 
restricts the possible side information sequences to only a small subset of the jointly typical set. More 
importantly, the CCSC protocol presented in [11] is non-constructive - a practical coding scheme 
does not follow immediately from it. In contrast, the scheme proposed in this paper addresses 
the broader problem of SCSI without putting any constraint on the size of the set of possible side 
information sequences. Moreover, the proposed approach clearly articulates the guideline to choose 
an appropriate channel code to design a practical SCSI scheme. 

After the publication of our preliminary work [2 , that first proposed the usage of list decoding for 
SCSI, Li and Ramamoorthy [19^ suggested using RS codes in the context of SCSI with feedback. They 
proposed using soft-decision list decoding for better performance and used simulation for constructive 
code design. However, we demonstrate that computationally complex soft-decision decoding is not 
justified in the SCSI setting. We also show that SCSI schemes can be constructed algebraically 
using RS codes and BCH codes by making maximum use of their known list decoding bounds. Not 
surprisingly, later in this paper, we demonstrate that our algebraic approach with hard-decision list 
decoding achieves the same compression as that achieved by the approach in jl9j . 

The organization of the rest of the paper is as follows. In Section [2] we briefly review the technique 
of syndrome source coding which provides a framework for designing SCSI schemes using linear block 
codes. In Section|3]we present the notion of list decoding and describe how this can be used to design 
a SCSI scheme based on the technique of syndrome source coding. The usage of Reed-Solomon codes 
and BCH codes in the design of practical constructive codes for SCSI is detailed in Section[4j Finally 
we conclude the paper in Section [5] 

2 Syndrome Source Coding for q-ary Symmetric Correlation 

The most challenging problem in the design of a practical binning scheme is the systematic construc- 
tion of bins with rich algebraic structures so that the bin indexing and typical set decoding can be 
performed with reasonable complexity. In this regard, there is a close connection between binning 
and channel coding. A linear block code induces a partitioning of the source space into cosets that 
can be indexed by their respective syndromes [23] . If the cosets of a linear code are such that each of 
them with high probability contains only one sequence from the typical set then the cosets effectively 
act as bins. In this case, the index of the bin in which a source sequence belongs can be computed 
as the syndrome of the sequence. 

Consider encoding a memoryless q-aiy source X with correlated side information Y available only 
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Figure 1: The source X and side information Y is correlated as Y — X + U where U is an IID q-ary 
source such that Pt[U = 0] = 1 — p and Pt[U = u] — p/{q — 1), for any u G GF{q) — {0}. Thus this 
correlation between X and Y is modelled via a g-ary symmetric channel with error probability p. 

at the decoder such that 

Y = X + U, 

where U is an IID q-ary source such that 

^ " ^ " I P/(1 - 9) if " e G^^(g) - {0} 

If Y were present at the encoder as well, X could be compressed at a rate of Hq{X\Y) = Hq(U) = 
—plog^p — {1 — p) logg(l — p) + plogq{q — 1) symbols. According to the Slepian-Wolf theorem, X 
can be compressed at the same rate Hq{U) even if Y is present only at the decoder. 

The above SCSI scenario can be modelled using a virtual f^-ary symmetric channel {qSC) with error 
probability p, which for a q-aiy symbol input, outputs the same input symbol with probability 
1 — p, or any of the other q — 1 symbols with probability p/{q— 1) (see Fig. [T]). This modelling of 
the correlation between the source and side information with a virtual channel allows us to use a 
channel code for the qSC to design a SCSI scheme [3J as described below. The capacity Cqsc of 
the qSC and equivalently of the additive noise channel is 

CqSC = l-Hq{U). 

According to the channel coding theorem [22], there exists a (n, k) linear block code C over GF{q) 
of rate k/n = R > {Cqsc — ^) such that the decoding error probability Pe < e for any e > and 
i5 > 0. Here the error event x ^ x corresponds to the fact that when the actual noise vector is u, 
the decoder decides the noise vector to be u with u ^ u. Therefore, Pr(x ^ x) = Pr(u ^ u). 

Now consider the following scheme of compression of X with the side information Y available only 
at the decoder based on this channel code. The encoding operation is to compute the (n — fc)- 
symbol syndrome Sx of the source sequence x € X" as Sx = Hx^, where H is the parity check 
matrix of C. If Cg^ denotes the coset corresponding to the syndrome Sx, then clearly x G Cg^. The 
decoding operation is to find the sequence x nearest (in Hamming distance) to the side information 
sequence y G Y" in the coset Cg^. Since Y = X + U, we can compute the syndrome of u as 
Su = Hy^ — Sx. Then the nearest neighbour decoding is equivalent to finding the minimum- weight 
noise vector u G Cs„ and decode x as x = y — u. Thus the probability of error of the scheme is 
Pr(x 7^ x) = Pr(u 7^ u), same as the channel decoding error, which tends to zero as n — > 00. This 
coding scheme, which can compress X at a rate of {n — k)/n < Hq{U) + 5 symbols with an arbitrarily 
small probability of error, is known as syndrome source coding [3 . Clearly, if a channel code of rate 
R = k/n is used for syndrome source coding, the achieved compression rate is {n — k) / n = 1 — R. 
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Figure 2: Unambiguous bounded distance decoding (BDD) is possible only if r < dmin/2 

3 Syndrome Source Coding Using List Decoding 

Clearly, the underlying linear block code C and its associated decoding algorithm impact the per- 
formance of a syndrome source coder. In the decoding of channel codes, the objective is to find the 
transmitted codeword c G C, given the received word r e GF{q)^. The natural decoding approach 
is to find the codeword which has the maximum likelihood of being transmitted given that r has 
been received. This approach known as maximum likelihood decoding (MLD) amounts to finding 
the codeword c closest to r in an appropriate measure of distance. However, MLD is known to 
be NP-complete in general [S]. In practice, bounded distance decoding (BDD), which has greatly 
reduced complexity, is preferred that ensures correct decoding only when the number of errors is 
upper bounded by some error correcting radius r. Obviously, an unambiguous BDD is possible only 
if T < d„iin/2 where dmin is the minimum distance of the code C (see Fig. [2|. 

Let us see the implication of BDD for channel coding and in turn for syndrome source coding. For 
a qSC with error probability p, the expected Hamming distance between the transmitted codeword 
c and the received word r is E[d{c, r)] = np. Thus for unambiguous decoding, we need a code 
C with dmin > 2np. This leads to the question of the largest rate possible with a code whose 
minimum distance is 2np. This remains as one of the biggest open questions in combinatorial coding 
theory [15) . However, it is known that the rate achievable with a block code of minimum distance 
2np is much less than 1 — Hq{p). For example, the best known upper bound on the rate achievable 
with a binary code of block length n, having a minimum distance of 2np, is less than 1 — H2{2p). 
Note that that the rate becomes when p > 0.25. This in turn implies that the compression rate 
achievable with syndrome source coding that relies on BDD is lower bounded by H2{2p), which is 
strictly larger than H2{p). 

In the above scenario, the main constraint is the requirement of unique decoding which sets the 
decoding radius to [(dmin ~ 1)/2J ■ One way to circumvent this limitation is to increase the decoding 
radius beyond [(dmin ^ 1)/2J and allow the decoder to output a list of codewords. This approach 
will be feasible as long as (i) the list contains only a small number of codewords and (ii) there is 
an effective way of extracting the correct codeword from the list. The method of decoding beyond 
[(c'min — 1)/2J is known as list decoding in the literature. Let Bq(r, e) denote the Hamming sphere 
of radius e around a point r in the space GF{qy\ A code C over GF{q) is said to be {p,L) list- 
decodable if \Mq{r,np) r\C\ < L. List decoding is considered feasible as long as L is a polynomial 
in n. 

To assess the feasibility of using list decoding for SCSI, let us look at the theoretical limits on list 
decoding. It has been shown in that for any integer L > 2, there exists a family of binary linear 
(p, L) list-decodable channel codes of rate R > 1 — H2{p) — 1/L. Thus allowing L to grow, a rate 
arbitrarily close to the theoretical limit 1 — H2{p) can be achieved. This in turns implies that the 
corresponding syndrome source coders have compression rate < H2{p) + which can be made 
arbitrarily close to the conditional entropy H2{p) by allowing L to grow. For a non-binary alphabet, 
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Figure 3: For a given side information sequence y G GF(qy\ the source sequence x e GF{qy\ with 
high probabihty, will be within the thin shell on the surface of Bg(y,np). Total number of points in 
the sheU is |B,(y,np)| < q'^^iip). 

a similar reasoning can be applied. It has been shown in [16J that for any alphabet size of g > 2, 
list size L > 2, and p E (0, 1 — there exists a family of {p, L) list-decodable q-ary linear channel 
codes of rate R> 1 — Hg{p) — 1/L. Consequently, the corresponding q-aiy syndrome source coders 
have a compression rate < Hq (p) + 1/L which approaches Hq (p) as the list size L increases. 

3.1 A geometrical interpretation of syndrome source coding using list 
decoding 

Consider the q-ary symmetric correlation between the source and side information with error proba- 
bility p. According to the law of large numbers, for large enough n, given a side information sequence 
y £ GF{qY\ the source sequence x e GF{q)" with high probability will be within a thin shell on 
the surface oi Mq{y,np). In fact, the thin shell corresponds to the set of sequences that are jointly 
typical with y. Now the total number of points in the shell is approximately equal to |Bg(y,np)| 
since for large n almost all the point in Bq(y, np) will be in the thin shell (see Fig.js]). It is known [H] 
that the number of points contained in Bq(y, np) is bounded by 

K(y,np)| <g"^'(f). (1) 

Now consider syndrome encoding of x using an (n, k) channel code C over GF{q). Clearly G induces 
a partitioning of the source data space GF{q)'^ into q^~^ cosets. Since the points in the shell are 
uniformly distributed over GF{q)'", for any syndrome s, the number of points in Bq(y, np) D Cg is 
approximately q^^i'^p) / q^~^ . From the list decoding point of view, this syndrome source coding is 
feasible if |Bg(y, np) n Cgj is small, which holds only ii R = k/n < 1 — Hq{p). 

3.2 Extracting the correct sequence from the list 

In syndrome source coding based on list decoding, the decoder essentially produces a list of sequences. 
Thus after list decoding we need to choose the correct sequence from the list. To aid the decoder 
in finding the correct sequence from the list, we propose to transmit a few CRC symbols along with 
the syndrome. A CRC-p code is defined by a generator polynomial g{£) of degree p that assigns 
p-symbol parity to a sequence. CRC codes are widely used in practice for error detection. In the 
setting of list-decoding based syndrome source coding, the use of a CRC code is expected to be 
effective for at least two reasons. Firstly, while in the context of channel coding, the CRC symbols 
are also subject to channel noise, this is not the case for syndrome source coding. In SCSI we are 
dealing with a virtual channel, modelling the correlation, instead of a real noisy channel. Thus in 
syndrome source coding, we can assume that these CRC symbols along with the syndrome will be 
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available to the decoder without error. Secondly, since the list size L is small (polynomial in n), only 
a few parity symbols are sufficient to correctly identify the desired sequence with high probability. 
Clearly a CRC-p code identifies the correct sequence from a list of size L with high probability if 
P > logg L. Thus a syndrome source coder based on list decoding that uses a CRC code to extract 
the correct sequence has a compression rate of Hq{p) + 1/L + log^ L/n. Since L is a polynomial in 
n, the compression rate Hq{p) + 1/L + log^ L/n approaches to Hq{p) as n grows. 

3.3 The proposed framework 

Given a linear {p,L) hst-decodable code C over GF{q) of rate > 1 — Hq{p), let us articulate the 
encoding and decoding operations involved in the syndrome coding of x in the presence of side 
information y only at the decoder. Let H be the parity check matrix of the code C. 

Encoding: The syndrome of the source sequence x is computed as Sx = Hx^. Let g{£^) be 
the generator polynomial of a CRC-p code and x{£,) be the polynomial of degree at most n — 1 
that corresponds to the sequence x. Then the p-symbol CRC of x corresponds to the polynomial 
h{£^) = x{(_) mod g(^) of degree at most p — 1. 

Decoding: First we need to list decode Cg^ considering the side information y as the received word. 
There can be two approaches. In the first approach, we compute the syndrome of the noise vector u 
as Su = Hy-^ — Sx. Then using a list decoding algorithm we determine the first L minimum- weight 
sequences {ui,U2,--- ,ul} from the coset Cs„. Subtracting each of the elements Ui,l < i < L, 
from y we get the list . The list Cg^ essentially consists of the sequences from Cg^ that are at 
a Hamming distance < np from y. In the second approach we first determine a sequence a S Cs^- 
Then given a e Cg^, the list decoding algorithm for C can be used to construct the list as follows. 
Compute y' = y — a. Using the list decoding algorithm for C, determine the list C consisting of 
those codewords of C that are within the Hamming sphere of radius np around y'. Then adding a 
to each of the codewords in £, we get the list Cg^ consisting of the sequences from Cg^ that are at 
a Hamming distance < np from y. 

Remarks: Between the two approaches, the first approach is preferred if the list decoding algorithm 
for C starts from the syndrome of the received word and compute the first L minimum weight 
sequences corresponding to that syndrome. Wu's list decoding algorithms for Reed-Solomon codes 
and BCH codes [JT] are examples of such algorithms. On the other hand, if the list decoding 
algorithm for C does not start from the syndrome, the second approach becomes usefuQ For 
example, the Guruswami-Sudan list decoding algorithm [17] for RS codes does not rely on syndrome 
computation. However, the problem of finding a sequence a S Cg^ amounts to solving the system of 
linear equations Ha-^ — s^. Since there are more unknown variables than equations, there is at least 
one non-zero solution to it. Thus we can find an a G Cg^ in polynomial time, for example, using 
Gaussian elimination. 

Once we construct the list £g^ using either of the two approaches, we pick the sequences x from 
the list such that x{£^) mod g{£,) = /i(^). Decoding is considered successful if there is only one such 
sequence. An error event occurs if none of the sequences matches the CRC h{^) or if there are more 
than one sequences having the same CRC h{^). 

We summarize the proposed framework in Fig. [4j Here it is worth noting that, the channel only 
models the correlation between the source and side information and the syndrome and CRC are not 
subject to noisy transmission. 



In fact, the second approach can be adopted with any hst decoding algorithm for C. 
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Figure 4: The proposed framework. 



4 Constructive Code Design 



Although the encoding and decoding algorithms presented in the previous section are theoretically 
sound, there are at least two challenges while designing codes for real- world applications. Firstly, the 
scheme assumes that d(x, y) « np with high probability, which holds when n oo. However, in the 
real world we have to operate with finite n. Secondly, the scheme also depends on the availability 
of a (p, L) list-decodable code with an efficient encoder and decoder. To date, efficient list decoding 
algorithms are known for the families of Reed-Solomon (RS), Bose-Chaudhuri-Hocquenghem (BCH), 
and Reed-Muller (RM) codes. Associated with each of these codes C is a known list decoding 
radius r. Before presenting the main results on these families of codes and their potential in syndrome 
source coding, in the following we provide a general guideline that can be used to design a practical 
SCSI scheme for a given g-ary symmetric correlation specified by the error probability p. 

Block length n: For better performance it is desirable to have n as large as possible. However, 
for large n the computational complexity may become impractical. While RS codes in practical 
applications are mostly of length n = 256 (due to the byte oriented world), it is feasible to go up to 
n = 1024 with binary BCH and RM codes. 

List decoding radius t: In theory, for n approaching cx), a list decoding radius of r < np + 5, 
where (5 > 0, is sufficient. In practice, the required list decoding radius t depends on the desired 
error rate e. For fixed n we need to have a list decoding radius of t > T^, where is such that 
Pr((i(x, y) > Te) < e. It can be shown that for large n we need to choose slightly bigger than 
np. Let e be a random variable representing (i(x,y). Then clearly e has a binomial distribution 
with mean np and variance {np{l — p)). For large n, with high probability e will be in the vicinity 
of np. For example, for n — 1000, p — 0.2, and e — 10^*, we have = 248. If we want to decrease 
the error probability to e = lO^'"^, we need to increase only by 8 to 256. 

Code rate R: Since the compression rate achieved with syndrome source coding based on a channel 
code of rate i? is 1 — i?, we need to pick a code of largest rate R whose list decoding radius is just 
larger than T^. 

CRC code generator g{^): There are a number of standard CRC codes, see [3D]. Since the list 
produced by the list decoder is guaranteed to be small, a few CRC symbols are enough to correctly 
extract the source sequence from the list with high probability. In practice, a 16-bit CRC is expected 
to be enough for n = 255 and for sources over GF{2^) this would incur only 0.78% overhead. 

In the following we discuss the main list decoding results for the families of RS codes and BCH codes 
and their implication for syndrome source coding. 
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4.1 Code design for non-binary sources using RS codes 

A (n, k) RS code C over GF{q) can be defined with its generator matrix G and parity check matrix 
H as follows [18] (here l<k<n^q-l) 

{G},,, ^a'l, i = 0, . . . , fc - 1, i = 0, . . . , n - 1 

{U},,j =a(*+i'^ i = 0,...,n- k-1, j = 0,...,n-l, 

where a is a primitive element of GF{q). According to the generator matrix, the codeword corre- 
sponding to a message u can be computed as c = {u{a^), u{a^), • • • , u{a"^^)). On the other hand, 
it follows from the parity check matrix that c e C if and only if c(a'+^) = for all < i < n — A; — 1. 
Further dmin = n — k + I since a (n, fc) RS code is maximum distance separable (MDS). 

A list decoding algorithm was first discovered for low rate RS codes by Sudan |57j and later improved 
and extended for all rates by Guruswami and Sudan !17] . For a RS code of rate R, the Guruswami- 
Sudan algorithm can correct up to n(l — VR) errors which is clearly beyond half of its minimum 
distance. The Guruswami- Sudan algorithm uses the polynomial representation corresponding to G. 
Given a received word r, the essential idea of the algorithm is to find the polynomials u(^) of degree 
at most k such that u{a^) = Vi for at least r values of j G [0,7i — 1]. Recently, building upon the 
Guruswami-Sudan approach, Wu [3T] has proposed an algorithm that can also achieve the same 
list decoding radius but with a reduced complexity. Moreover, the Wu algorithm, which relies on 
polynomial representation corresponding to H, is well suited for syndrome source coding since its 
decoding algorithm, like the Berlekamp-Massey algorithm, starts with syndromes. 

Example: Let us design a SCSI scheme for a source over GF{q = 2®) where the correlation between 
the source and side-information is (?-ary symmetric with error probability p = 0.3. Given the desired 
error rate of e = 10^^ and block length of n = 255, the value of turns out to be = 105. Thus 
we need a channel code of block length n = 255 having list decoding radius r > 105. Using the fact 
that the RS code of rate R has r = n(l — VR), we find that the (255, 88) is the desired RS code. 
Thus the compression rate achieved with this scheme is 1 — i? = 0.6549. When 16 GRC bits are 
considered, the compression rate increases to 0.6627. In contrast, with classical unique decoding we 
require a code to have dj^in = 2Tg -I- 1 = 211. It is the (255,45) RS code that has dmin = 211. The 
use of this code for syndrome source coding with unique decoding can only achieve a compression 
rate of 0.8235. 

4.2 Non-binary SCSI with feedback using RS codes 

Here we consider constructive code design for SCSI with feedback. In this setting, it is assumed that 
the decoder can request additional symbols via a feedback channel if the decoding is unsuccessful. A 
SCSI scheme can take advantage of the feedback channel as follows. It first uses a high rate channel 
code Ci to induce the partition and transmits the syndrome HiX"^. Upon receiving the syndrome, 
the decoder tries to decode. The decoder then informs the encoder whether the decoding is successful 
via the feedback channel. If the decoding is unsuccessful, the encoder chooses a lower rate code C2 
(increase the length of the syndrome) and transmits the syndrome H2X"^. The encoder gradually 
increases the transmission rate by using channel codes of lower rates until the decoding is successful. 

In the setting of SCSI with feedback, the proposed list decoding approach based on RS codes is 
suitable for two reasons. Firstly, the CRC symbols transmitted along with the syndrome provide 
an effective way of detecting whether the decoding is successful. Clearly the CRC symbols need 
to be transmitted only once at the beginning of the process. Secondly, as pointed out in |19j . the 
class of RS codes provides a natural rate adaptivity which stems from the nested structure of the 
parity check matrices of RS codes. Let Hi and H2 be the parity check matrices of RS code of rates 
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Figure 5: Compression rate achievable with Ust decoding and unique decoding of RS codes under 
g-ary symmetric correlation specified by the error probability p. Here q = 2^ and e = 10^'^. 



^1 = ki/n and i?2 = k2/n respectively, where fci > fc2- Let the encoder first transmit the n — ki 
symbol syndrome = HiX"^ and suppose that the decoding is unsuccessful. Now if the encoder 
chooses the lower rate code for syndrome source coding, it needs to inform the decoder of the n — k2 
symbol syndrome — H2X^. Since Hi is embedded in H2 as its first n — ki rows, the encoder 
can compute and transmit the fci — k2 additional syndromes as the inner product of x and the last 
ki — ^2 rows of H2 . 

Example: Let us design a SCSI scheme for a source over GF{q = 2^) where the correlation between 
the source and side-information can be modelled with a g-ary symmetric channel with error proba- 
bility p = 0.34. Since q = 2*, we choose the block length a.s n — q — I = 255. Let the desired error 
rate is e = 10~^. Let e denotes the Hamming distance between the source sequence x G GF{q)^^^ 
and the side-information sequence y S GF{q)^^^. We first determine I = 3 and h = 171 such that 
Pr[Z < e < h] > 1 — e. Let C" denotes the RS code of largest rate having list decoding radius t > i. 
Then clearly C' D (7'+^ D • • • D C*. We start the encoding with which is the (255, 249) RS code. 
If the decoding is unsuccessful we choose which is the (255, 247) RS code having list decoding 
radius r > 4. Thus we gradually increase the list decoding radius by decreasing the rate of the 
corresponding RS code when the decoding is unsuccessful. We declare a failure if the usage of C^^^, 
which is the (255, 27) RS code, fails to provide correct decoding. Clearly the average compression 
rate achieved with this scheme is — ^]{^~ = 0.5634 symbols where denotes the rate 

of C\ 

In Fig. [5j we present the compression rates achievable with RS codes for sources over GF{2^). Here 
we have used the maximum block length n = 255 and assumed e = 10~^ as the desired error rate. 
Clearly the SCSI scheme using list decoding of RS codes outperforms the scheme that relies on 
classical unique decoding. It is worth noting that for p > 0.4, no compression can be achieved with 
unique decoding. For example, when p = 0.41, the value of turns out to be 129. With classical 
unique decoding to achieve a decoding error rate of < 10"'^ we need a RS code of minimum distance 
c^min = 259 which does not exist for n = 255. It also follows from the figure that the compression 
rate can be reduced further if a feedback channel is available to inform the encoder whether the 
decoding is successful. 

Soft-decision vs hard-decision decoding: In the SCSI setting the channel only models the correlation 
between the source and side-information and thus the processes of modulation and demodulation 
are not involved. Therefore, we argue that the computationally complex soft-decision decoding of 



10 



Without teedback 

- ■ - ■ With teedback 



3 - 




0.5 - 



0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 

Agreement probability, 1 - p 

Figure 6: The gap between H{X\Y) and the transmission rate achievable with list decoding ol RS 
codes for both the SCSI settings: classical (without feedback) and with feedback. Here q ~ 2^ and 
e= 10-3. 

RS codes, used for syndrome source coding, does not yield better performance over its hard-decision 
counterparts. To validate this we compare our results with that reported by Li and Ramamoor- 
thy in T^. Li and Ramamoorthy, given a q-ary {q = 2*) correlation model specified by the error 
probability p, determined the required transmission rate to achieve an error rate of e = 10"'^ via 
simulation. In simulation, Li and Ramamoorthy used soft-decision decoding of the corresponding 
RS codes. In |19j . the simulation results have been reported as the gaps between the actual trans- 
mission rates and H{X\Y) at different agreement probabilities (1 — p). In Fig. |6j we also present 
the gap between H{X\Y) and the transmission rate calculated algebraically at different agreement 
probabilities 1— p. Comparing Fig. [6] with that reported in jl9j we find that our algebraic approach 
yields the same rate as that achieved using simulation and soft-decision decoding. 

4.3 Code design for binary sources using BCH codes 

Binary BCH codes can be interpreted as binary sub-codes of RS codes [33], i.e., if Cjis is an RS 
code over GF(2™), then Crs(^GF{2)"- is a BCH code. This interpretation allows Wu's list decoding 
algorithm for RS codes to be used for the list decoding of BCH codes. In [3T] Wu also presents an 
improved algorithm for list decoding of binary BCH codes that achieves a list decoding radius of 
T = f (1 — \/l — 2D), where D is the designed relative distance of the BCH code [20]. 

Example: Consider designing a SCSI scheme for a q-ary symmetric correlation with error probability 
p = 0.2 and e = 10"''. As binary BCH codes of length up to 1023 can be implemented without any 
difficulty, we choose n = 1023. For the given values of n, p, and e, we calculate = 254. The 
(1023, 56) BCH code with D > 0.3743 has r > 382 and thus can achieve an error probability of 
Pe < lO"* if used for syndrome source coding. The compression rate of this scheme is 0.9453 which 
slightly increases to 0.9570 when 12 CRC bits are considered. Note that with unique decoding it 
would need a code of dmin > 508. The BCH code of designed distance > 508 is the (1023, 11) code 
which only achieves a compression rate of 0.9892. 

In Fig. [Tj we present the compression rates achievable with BCH codes for binary sources. Here we 
have used BCH codes of block length n ~ 1023 and assumed e ~ 10^^ as the desired error rate. 
Clearly the SCSI scheme using list decoding of BCH codes outperforms the scheme that relies on 
classical unique decoding. It is known that for binary sources no compression can be achieved with 
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Figure 7: Compression rate achievable with list decoding and unique decoding of BCH codes under 
binary symmetric correlation specified by the crossover probability p for a desired error rate of 
e < 10-3. 

classical unique decoding if p > 0.25. It is worth noting that with BCH codes no compression can 
be achieved with unique decoding ii p > 0.20. For example, when p — 0.21, the value of turns 
out to be 256. With classical unique decoding to achieve a decoding error rate of < 10"'^ we need 
a BCH code having a designed error correcting capability oi t > 256. However, among all the BCH 
codes of block length n = 1023, the (1023, 11) BCH code has the maximum designed error correcting 
capability of t = 255. Thus we can not achieve any compression using unique decoding of BCH 
codes when p > 0.20. 

5 Conclusions 

In SCSI it is customary to model the correlation between the source and side information via a 
virtual channel. In this paper, we have recognized two important aspects of this virtual channel. 
Firstly, the compressed symbols are not transmitted over a real noisy channel in the setting of SCSI. 
We exploit this advantage by transmitting a few additional symbols to accomplish selection of the 
correct data from the list obtained by the list decoder. These additional symbols are provided by 
a CRC code. Secondly, we observe that no modulation and demodulation processes are involved in 
SCSI and thus computationally complex soft-decision decoding is not justified over its hard-decision 
counterparts. 

We show that our list decoding based source coding has a compression rate that is significantly 
lower than a classical unique decoding based source coder. Moreover, the proposed approach has 
the advantage that given the conditional entropy and a class of list decodable code, it allows for an 
algebraic approach to choosing the best channel code (without resorting to simulation) that achieves 
the desired error rate. Our future works aim at extending this list-decoding based SCSI scheme to 
the problem of compression of multiple correlated sources along the line of the approach in [H [3 |S] . 
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